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1 Introduction 


Expon ential smoot h ers provid e an efficient way to co mpute moving averages of 
signals iTond [l990j| ; iBox et aP [2004 1 ; iLutkepoij 2010l| . This can be particularly 
useful for real time applications. Here we define, describe and analyze exponential 
smoothing algorithms, with an eye towards practical applications. 


Let Xt ^, Xt 2 , • • • be a sequence of random variables observed at times ti <t 2 < 

• • • be the times at which the variables are observed. We define the exponentially 
smoothed average Xt evaluated at time t as follows 


II 

'1^ 

(1) 

Wt 

Xt=Y, 

(2) 

t,<t 


Wt = J2 

(3) 

t,<t 



where r > 0 is known as the time scale parameter. 


2 Recursive Update Equations 

A key advantage of exponential smoothing is that it does not require storage of 
past observations in memory. The current smoothed average, the current new ob¬ 
servation, the time of the last observation and the time of the current observation 
determine the new smoothed value. 


Version 1: Note 

w, = Xt, + + • • • + Xt, 

where S{k,j) = tk — tj. Moreover 

Xt,_, = Xt,_, + + • • • + 


(4) 

(5) 


Thus 


^tk — Xt, + at,Xtk_, 

at, = 


( 6 ) 

(7) 


with initial condition Xt, = Xi. Similarly 

wtk = wt, + at, ( 8 ) 

with initial condition wt, =1. If t is not one of the sampling times, i.e., t G {tk, tk+i) 
then 


At = 

At = 5 = Xt, 

Wt 


(9) 

( 10 ) 


( 11 ) 
















Version 2 ; We shall see later that the previous update equation may run into 
numerical problems as a —>■ 0. Here we see a version of the update equations that 
avoids this problem, though it runs into other issues as a —>■ 1. The trick is to 
multiply the numerator and denominator of (HD by 1 — a, which is acceptable as 
long as a < 1. Regarding the numerator of ([T|), let 


Xt, = (1 - aM, = (1 - aM, 

+ (1 “ cttk)(^tkXtk-i 
= (1 - ottk)Xti, + 

= ^tk + (1 - Oitk){Xtk - Xtk-i) 

Regarding the denominator of ([T|) let 

Wtk =(1 - Oitk)wtk = (1 - OtJ + (1 - atk)atkWtk_, 
= i^ - atk) + atkWtk_, 

=1 + {1 - atk){l - wtk_,) 

Then 

Xtk {l-atk)Xt Xtk 
wtk {l-atk)wt wtk 


( 12 ) 

(13) 


(14) 


(15) 


3 Constant Sampling Rate 

It is useful to analyze how the update equations behave when the sampling rate is 
constant. In this case the time interval between observations is a constant S and 
thus a = is also constant. 


Update Equations Version 1: Consider update equations of the form 


Uk+i = c + auk 

(16) 

where c is a constant. At equilibrium 


Uk = c + auk 

(17) 

c 

Uk = , 

1 — a 

(18) 

Thus at equilibrium 


wtk=l + awtk-i 

(19) 

1 

= 1 

1 — a 

(20) 

Note that Wtk can grow very large for a —>■ 1, potentially causing numerical prob¬ 
lems. If the Xtk variables are independent identically distributed random variables 
with mean /i and variance then the equilibrium equations for the expected value 

are as follow: 


E[Xt,]=^^ + aE[Xt,\ 

(21) 


(22) 

E[Xt,] = = M 

(23) 







and the equilibrium equations for the variance are as follow 

Var[Xt^] = cr^ + a‘^Var[Xtk] 

2 

Var[Xt^] = ^ 

I — 

,, tv ^ (1-a)^ 2 1-a 2 

Var[Xt^] = - -^cr = -r—cr 

1 — 1 + a 

Update equations Version 2: In this case 

Xt, = (1 - a)Xt, + aXt,., = Xt, + (1 - a){Xt, - Xt,.J 
wtk = (!-«) + = 1 + (1 - a)(l - J 

The equilibrium equation for w is as follows 

wtk = (1 - o) + awt^ 

Thus at equilibrium 

Wtt. = 1 

This can be also seen using the sum of geometric series 

n 1 2 ^_1 1 O 

H-ho = -- 


wt, = l- (33) 

lim wt^ = 1 (34) 

k—¥oo 

Thus, asymptotically, as —>■ oo 

Xt, = aXt, + (1 - a)Xt,_, = Xt, + (1 - a){Xt, - (35) 

If the observations are independent identically distributed with mean fj, and variance 
then the asymptotic equations for the mean and variance of the smoother are as 
follow 

E[Xt] = + (1 - a)E[Xt] (36) 

Var[Xt] = + (I - a^)Var[Xt] (37) 

Thus, asymptotically, as tfe —>■ oo 

E[^tJ = — =^i (38) 

a 


Var[VtJ = ^ —^0-2 = (39) 

1 — 0^ 1 + Of 

Effective Number of Averaged Observations: If X were the average of n 
independent observations the variance of X would be jn. Thus asymptotically, 
the effective number of observations averaged by the exponential smoother can be 
defined as follows 


n = 

a = 


1 T o 
1 — a 
n — 1 
n + 1 


(40) 

(41) 


Thus, for a = 0 we average one observation. The number of averaged observations 
increases unboundedly as a increases. 



Effective Time Window: In a time window of size T we 
Thus 

get Tjd observations. 

T/d+l T + 5 

(42) 

Taking logs 


-- =log(T-(5)-log(T + J) 
r 

(43) 

- = log(T + 6) - log(T - (5) 
r 

(44) 

1 log(T -1- (5) - log(T - (5) 
r S 

(45) 

Thus, in the limit as 5 ^ 0 


1 _2^1og(T) _ 2 

T dT T 

(46) 

T 

^ “ "2 

(47) 

i.e., the effective window size T is twice the time scale parameter r. Turns out for 

J < 0.1, the approximation 


\og{T + 6) - \og{T - 6) _ 2 
(5 ^ T 

(48) 

is quite good. Under this approximation we get 


A/ 2 

SIt= — = - 

1 n 

(49) 

a = e-^!^ = e-2/” 

(50) 

n = 2 log(a“^) 

(51) 


4 Alternative Approaches 

Kalman filters provide an alternative way to deal with variable interval observa¬ 
tions. In the constant interval case Kalman filters and exponential smoothers are 
asymptotically equivalent. In the variable interval case they both weight less the 
last smooth value the longer the time between the last observation and the current 
observation. However they differ in the way they do so. An advantage of expo¬ 
nential smoothers over Kalman filters is their simplicity. An advantage of Kalman 
filters is that they provide estimates of the uncertainty of the smoothed values. This 
can be useful in some situations. 
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